Method to compute sliding window block sum using instruction based selective horizontal addition in vector processor

ABSTRACT

Disclosed techniques relate to forming a block sum of picture elements employing a vector dot product instruction to sum packed picture elements and the mask producing a vector of masked horizontal picture element. The block sum is formed from plural horizontal sums via vector single instruction multiple data (SIMD) addition.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.16/291,405 filed on Mar. 4, 2019, which is a continuation of U.S. patentapplication Ser. No. 14/391,222 filed on Nov. 3, 2015, which claimspriority to Indian Provisional Application No. 5508/CHE/2014 filed onNov. 3, 2014, all of which are incorporated herein by reference.

TECHNICAL FIELD

The technical field of this invention is digital data processing andmore specifically computing a sliding window block sum.

BACKGROUND

A two dimensional Block Sum computation is performed by summation ofevery element contained in a block of size m×n which lies within amatrix of size M×N, where M>m and N>n. When the block sum is computedfor a window of size m×n around every element of a matrix of size M×N,creating a new matrix of dimensions (M−m+1)×(N−n+1) replacing everyelement of the original matrix with the block sum of the window aroundit, this is called a sliding window block sum computation.

Sliding window block sum computation is an important common step in manykey low level vision kernels. In the Harris Corner Detection algorithm(described in C. Harris and M. Stephens, “A Combined Corner and EdgeDetector,” Alvey Vision Conference, 1988), the block sum of squares ofpixel intensity gradients of a sub-window around every pixel needs to becomputed for identifying the sub-window which is potentially, a goodcorner. Thus this block sum of squares of pixel intensity gradients is agood feature to track. Similarly in a ORB feature detection anddescription algorithm (E. Rublee, V. Rabaud, K. Konolige, G. Bradski,“ORB: An Efficient Alternative to SIFT or SURF,” ICCV, 2564-2571, 2011),every pixel in the window region around an identified feature issmoothened by substituting a 5×5 block sum around that pixel. Suchexamples of sliding window block sum calculations are numerous inembedded vision space.

Given the importance of sliding window block sum computation in visionapplications, a fast technique to compute block sums for a slidingwindow would speed up performance of many vision kernels. Since visionalgorithms typically involve similar computation tasks across huge imageblocks or across the entire image and also need to operate at highframes per second (FPS). Vector single instruction multiple data (SIMD)engines are best suited for solving vision tasks. In these applicationshigh capacity vector processing can boost performance.

SUMMARY

This invention forms a block sum of picture elements employing a vectordot product instruction to sum packed picture elements and the maskproducing a vector of masked horizontal picture element. The block sumis formed from plural horizontal sums via vector single instructionmultiple data (SIMD) addition.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of this invention are illustrated in thedrawings, in which:

FIG. 1 illustrates a dual scalar/vector datapath processor according toone embodiment of this invention;

FIG. 2 illustrates registers and functional units in the dualscalar/vector datapath processor illustrated in FIG. 1;

FIG. 3 illustrates a global scalar register file;

FIG. 4 illustrates a local scalar register file shared by arithmeticfunctional units;

FIG. 5 illustrates a local scalar register file shared by multiplyfunctional units;

FIG. 6 illustrates a local scalar register file shared by load/storeunits;

FIG. 7 illustrates a global vector register file;

FIG. 8 illustrates a predicate register file;

FIG. 9 illustrates a local vector register file shared by arithmeticfunctional units;

FIG. 10 illustrates a local vector register file shared by multiply andcorrelation functional units;

FIG. 11 illustrates pipeline phases of a central processing unitaccording to an embodiment of this invention;

FIG. 12 illustrates sixteen instructions of a single fetch packet;

FIG. 13 illustrates an example of the instruction coding of instructionsused by this invention;

FIG. 14 illustrates the bit coding of a condition code extension slot 0;

FIG. 15 illustrates the bit coding of a condition code extension slot 1;

FIG. 16 illustrates the bit coding of a constant extension slot 0;

FIG. 17 is a partial block diagram illustrating constant extension;

FIG. 18 illustrates the carry control for SIMD operations according tothis invention;

FIG. 19 illustrates an example of a sliding window sum computation ofthe prior art;

FIG. 20 illustrates one aspect of such a sliding window sum including aSIMD vector sum operation;

FIG. 21 illustrates an exemplary 3 by 3 window with individual elementslabeled for ease of reference;

FIG. 22 illustrates an order of disposition of the elements A through Iin memory;

FIG. 23 illustrates a second vector load of an example operation;

FIG. 24 illustrates a third vector load of an example operation;

FIG. 25 illustrates the values of a running sum upon initialization, atintermediate steps and the final value;

FIG. 26 is a flow chart outlining a prior art technique;

FIG. 27 illustrates the relationship between element values in an imageand a corresponding integral image;

FIG. 28 illustrates a way of calculating the sum of pixel values in agiven image from the integral values;

FIG. 29 schematically illustrates the operation of the VDOTPMPNinstruction;

FIG. 30 is a flow chart illustrating the process of this invention;

FIG. 31 illustrates using a VDOTPMPN instruction for a horizontal sum ofan example having an eight element vector length for a block sum of a3×3 block; and

FIG. 32 illustrates using SIMD add for a vertical sum in the 3×3 blocksum example.

DETAILED DESCRIPTION

FIG. 1 illustrates a dual scalar/vector datapath processor according toa preferred embodiment of this invention. Processor 100 includesseparate level one instruction cache (L1I) 121 and level one data cache(L1D) 123. Processor 100 includes a level two combined instruction/datacache (L2) 130 that holds both instructions and data. FIG. 1 illustratesconnection between level one instruction cache 121 and level twocombined instruction/data cache 130 (bus 142). FIG. 1 illustratesconnection between level one data cache 123 and level two combinedinstruction/data cache 130 (bus 145). In the preferred embodiment ofprocessor 100 level two combined instruction/data cache 130 stores bothinstructions to back up level one instruction cache 121 and data to backup level one data cache 123. In the preferred embodiment level twocombined instruction/data cache 130 is further connected to higher levelcache and/or main memory in a manner not illustrated in FIG. 1. In thepreferred embodiment central processing unit core 110, level oneinstruction cache 121, level one data cache 123 and level two combinedinstruction/data cache 130 are formed on a single integrated circuit.This signal integrated circuit optionally includes other circuits.

Central processing unit core 110 fetches instructions from level oneinstruction cache 121 as controlled by instruction fetch unit 111.Instruction fetch unit 111 determines the next instructions to beexecuted and recalls a fetch packet sized set of such instructions. Thenature and size of fetch packets are further detailed below. As known inthe art, instructions are directly fetched from level one instructioncache 121 upon a cache hit (if these instructions are stored in levelone instruction cache 121). Upon a cache miss (the specified instructionfetch packet is not stored in level one instruction cache 121), theseinstructions are sought in level two combined cache 130. In thepreferred embodiment the size of a cache line in level one instructioncache 121 equals the size of a fetch packet. The memory locations ofthese instructions are either a hit in level two combined cache 130 or amiss. A hit is serviced from level two combined cache 130. A miss isserviced from a higher level of cache (not illustrated) or from mainmemory (not illustrated). As is known in the art, the requestedinstruction may be simultaneously supplied to both level one instructioncache 121 and central processing unit core 110 to speed use.

Instruction dispatch unit 112 determines the target functional unit ofeach fetched instruction. Instruction dispatch unit 112 directs eachinstruction to its target functional unit. In the preferred embodimentof this invention, central processing unit core 110 includes pluralfunctional units to perform instruction specified data processing tasks.The functional unit assigned to an instruction is completely specifiedby the instruction produced by a compiler. The hardware of centralprocessing unit core 110 has no part in this functional unit assignment.In the preferred embodiment instruction dispatch unit 112 may operate onplural instructions in parallel. The number of such parallelinstructions is set by the size of an execute packet including theinstructions. This will be further detailed below.

One part of the dispatch task of instruction dispatch unit 112 isdetermining whether the instruction is to execute on a functional unitin scalar datapath side A 115 or vector datapath side B 116. Aninstruction bit within each instruction called the s bit determineswhich datapath the instruction controls. This will be further detailedbelow.

Instruction decode unit 113 decodes each instruction in a currentexecute packet. Decoding includes identification of the functional unitperforming the instruction, identification of registers used to supplydata for the corresponding data processing operation from among possibleregister files and identification of the register destination of theresults of the corresponding data processing operation. The result ofthis decoding is signals for control of the target functional unit toperform the data processing operation specified by the correspondinginstruction.

Central processing unit core 110 includes control registers 114. Controlregisters 114 store information for control of the functional units inscalar datapath side A 115 and vector datapath side B 116. Thisinformation could be mode information or the like.

The decoded instructions from instruction decode 113 and informationstored in control registers 114 are supplied to scalar datapath side A115 and vector datapath side B 116. As a result functional units withinscalar datapath side A 115 and vector datapath side B 116 performinstruction specified data processing operations upon instructionspecified data and store the results in an instruction specified dataregister or registers. Each of scalar datapath side A 115 and vectordatapath side B 116 include plural functional units that preferablyoperate in parallel. These will be further detailed below in conjunctionwith FIG. 2. There is a datapath 117 between scalar datapath side A 115and vector datapath side B 116 permitting data exchange.

Central processing unit core 110 includes further non-instruction basedmodules. Emulation unit 118 permits determination of the machine stateof central processing unit core 110 in response to instructions. Thiscapability will typically be employed for algorithmic development.Interrupts/exceptions unit 119 enable central processing unit core 110to be responsive to external, asynchronous events (interrupts) and torespond to attempts to perform improper operations (exceptions).

Central processing unit core 110 includes streaming engine 125.Streaming engine 125 supplies two data streams from predeterminedaddresses typically cached in level two combined cache 130 to operandinputs of functional units of vector datapath side B. This providescontrolled data movement from memory (as cached in level two combinedcache 130) directly to operand inputs. This is further detailed below.

FIG. 1 illustrates exemplary data widths of busses between variousparts. Level one instruction cache 121 supplies instructions toinstruction fetch unit 111 via bus 141. Bus 141 is preferably a 512-bitbus. Bus 141 is unidirectional from level one instruction cache 121 tocentral processing unit 110. Level two combined cache 130 suppliesinstructions to level one instruction cache 121 via bus 142. Bus 142 ispreferably a 512-bit bus. Bus 142 is unidirectional from level twocombined cache 130 to level one instruction cache 121.

Level one data cache 123 exchanges data with register files in scalardatapath side A 115 via bus 143. Bus 143 is preferably a 64-bit bus.Level one data cache 123 exchanges data with register files in vectordatapath side B 116 via bus 144. Bus 144 is preferably a 512-bit bus.Busses 143 and 144 are illustrated as bidirectional supporting bothcentral processing unit 110 data reads and data writes. Level one datacache 123 exchanges data with level two combined cache 130 via bus 145.Bus 145 is preferably a 512-bit bus. Bus 145 is illustrated asbidirectional supporting cache service for both central processing unit110 data reads and data writes.

Level two combined cache 130 supplies data of a first data stream tostreaming engine 125 via bus 146. Bus 146 is preferably a 512-bit bus.Streaming engine 125 supplies data of this first data stream to operandinputs of functional units of vector datapath side B 116 via bus 147.Bus 147 is preferably a 512-bit bus. Level two combined cache 130supplies data of a second data stream to streaming engine 125 via bus148. Bus 148 is preferably a 512-bit bus. Streaming engine 125 suppliesdata of this second data stream to operand inputs of functional units ofvector datapath side B 116 via bus 149. Bus 149 is preferably a 512-bitbus. Busses 146, 147, 148 and 149 are illustrated as unidirectional fromlevel two combined cache 130 to streaming engine 125 and to vectordatapath side B 116 in accordance with the preferred embodiment of thisinvention.

FIG. 2 illustrates further details of functional units and registerfiles within scalar datapath side A 115 and vector datapath side B 116.Scalar datapath side A 115 includes global scalar register file 211,L1/S1 local register file 212, M1/N1 local register file 213 and D1/D2local register file 214. Scalar datapath side A 115 includes L1 unit221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225 and D2 unit 226.Vector datapath side B 116 includes global vector register file 231,L2/S2 local register file 232, M2/N2/C local register file 233 andpredicate register file 234. Vector datapath side B 116 includes L2 unit241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245 and P unit 246.There are limitations upon which functional units may read from or writeto which register files. These will be detailed below.

Scalar datapath side A 115 includes L1 unit 221. L1 unit 221 generallyaccepts two 64-bit operands and produces one 64-bit result. The twooperands are each recalled from an instruction specified register ineither global scalar register file 211 or L1/S1 local register file 212.L1 unit 221 preferably performs the following instruction selectedoperations: 64-bit add/subtract operations; 32-bit min/max operations;8-bit Single Instruction Multiple Data (SIMD) instructions such as sumof absolute value, minimum and maximum determinations; circular min/maxoperations; and various move operations between register files. Theresult may be written into an instruction specified register of globalscalar register file 211, L1/S1 local register file 212, M1/N1 localregister file 213 or D1/D2 local register file 214.

Scalar datapath side A 115 includes S1 unit 222. S1 unit 222 generallyaccepts two 64-bit operands and produces one 64-bit result. The twooperands are each recalled from an instruction specified register ineither global scalar register file 211 or L1/S1 local register file 212.S1 unit 222 preferably performs the same type operations as L1 unit 221.There optionally may be slight variations between the data processingoperations supported by L1 unit 221 and S1 unit 222. The result may bewritten into an instruction specified register of global scalar registerfile 211, L1/S1 local register file 212, M1/N1 local register file 213or D1/D2 local register file 214.

Scalar datapath side A 115 includes M1 unit 223. M1 unit 223 generallyaccepts two 64-bit operands and produces one 64-bit result. The twooperands are each recalled from an instruction specified register ineither global scalar register file 211 or M1/N1 local register file 213.M1 unit 223 preferably performs the following instruction selectedoperations: 8-bit multiply operations; complex dot product operations;32-bit bit count operations; complex conjugate multiply operations; andbit-wise Logical Operations, moves, adds and subtracts. The result maybe written into an instruction specified register of global scalarregister file 211, L1/S1 local register file 212, M1/N1 local registerfile 213 or D1/D2 local register file 214.

Scalar datapath side A 115 includes N1 unit 224. N1 unit 224 generallyaccepts two 64-bit operands and produces one 64-bit result. The twooperands are each recalled from an instruction specified register ineither global scalar register file 211 or M1/N1 local register file 213.N1 unit 224 preferably performs the same type operations as M1 unit 223.There may be certain double operations (called dual issued instructions)that employ both the M1 unit 223 and the N1 unit 224 together. Theresult may be written into an instruction specified register of globalscalar register file 211, L1/S1 local register file 212, M1/N1 localregister file 213 or D1/D2 local register file 214.

Scalar datapath side A 115 includes D1 unit 225 and D2 unit 226. D1 unit225 and D2 unit 226 generally each accept two 64-bit operands and eachproduce one 64-bit result. D1 unit 225 and D2 unit 226 generally performaddress calculations and corresponding load and store operations. D1unit 225 is used for scalar loads and stores of 64 bits. D2 unit 226 isused for vector loads and stores of 512 bits. D1 unit 225 and D2 unit226 preferably also perform: swapping, pack and unpack on the load andstore data; 64-bit SIMD arithmetic operations; and 64-bit bit-wiselogical operations. D1/D2 local register file 214 will generally storebase and offset addresses used in address calculations for thecorresponding loads and stores. The two operands are each recalled froman instruction specified register in either global scalar register file211 or D1/D2 local register file 214. The calculated result may bewritten into an instruction specified register of global scalar registerfile 211, L1/S1 local register file 212, M1/N1 local register file 213or D1/D2 local register file 214.

Vector datapath side B 116 includes L2 unit 241. L2 unit 241 generallyaccepts two 512-bit operands and produces one 512-bit result. The twooperands are each recalled from an instruction specified register ineither global vector register file 231, L2/S2 local register file 232 orpredicate register file 234. L2 unit 241 preferably performs instructionsimilar to L1 unit 221 except on wider 512-bit data. The result may bewritten into an instruction specified register of global vector registerfile 231, L2/S2 local register file 232, M2/N2/C local register file 233or predicate register file 234.

Vector datapath side B 116 includes S2 unit 242. S2 unit 242 generallyaccepts two 512-bit operands and produces one 512-bit result. The twooperands are each recalled from an instruction specified register ineither global vector register file 231, L2/S2 local register file 232 orpredicate register file 234. S2 unit 242 preferably performsinstructions similar to S1 unit 222 except on wider 512-bit data. Theresult may be written into an instruction specified register of globalvector register file 231, L2/S2 local register file 232, M2/N2/C localregister file 233 or predicate register file 234.

Vector datapath side B 116 includes M2 unit 243. M2 unit 243 generallyaccepts two 512-bit operands and produces one 512-bit result. The twooperands are each recalled from an instruction specified register ineither global vector register file 231 or M2/N2/C local register file233. M2 unit 243 preferably performs instructions similar to M1 unit 223except on wider 512-bit data. The result may be written into aninstruction specified register of global vector register file 231, L2/S2local register file 232 or M2/N2/C local register file 233.

Vector datapath side B 116 includes N2 unit 244. N2 unit 244 generallyaccepts two 512-bit operands and produces one 512-bit result. The twooperands are each recalled from an instruction specified register ineither global vector register file 231 or M2/N2/C local register file233. N2 unit 244 preferably performs the same type operations as M2 unit243. There may be certain double operations (called dual issuedinstructions) that employ both M2 unit 243 and the N2 unit 244 together.The result may be written into an instruction specified register ofglobal vector register file 231, L2/S2 local register file 232 orM2/N2/C local register file 233.

Vector datapath side B 116 includes C unit 245. C unit 245 generallyaccepts two 512-bit operands and produces one 512-bit result. The twooperands are each recalled from an instruction specified register ineither global vector register file 231 or M2/N2/C local register file233. C unit 245 preferably performs: “Rake” and “Search” instructions;up to 512 2-bit PN*8-bit multiplies I/Q complex multiplies per clockcycle; 8-bit and 16-bit Sum-of-Absolute-Difference (SAD) calculations,up to 512 SADs per clock cycle; horizontal add and horizontal min/maxinstructions; and vector permutes instructions. C unit 245 also contains4 vector control registers (CUCR0 to CUCR3) used to control certainoperations of C unit 245 instructions. Control registers CUCR0 to CUCR3are used as operands in certain C unit 245 operations. Control registersCUCR0 to CUCR3 are preferably used: in control of a general permutationinstruction (VPERM); and as masks for SIMD multiple DOT productoperations (DOTPM) and SIMD multiple Sum-of-Absolute-Difference (SAD)operations. Control register CUCR0 is preferably used to store thepolynomials for Galois Field Multiply operations (GFMPY). Controlregister CUCR1 is preferably used to store the Galois field polynomialgenerator function.

Vector datapath side B 116 includes P unit 246. P unit 246 performsbasic logic operations on registers of local predicate register file234. P unit 246 has direct access to read from and write to predicationregister file 234. These operations include AND, ANDN, OR, XOR, NOR,BITR, NEG, SET, BITCNT, RMBD, BIT Decimate and Expand. A commonlyexpected use of P unit 246 includes manipulation of the SIMD vectorcomparison results for use in control of a further SIMD vectoroperation.

FIG. 3 illustrates global scalar register file 211. There are 16independent 64-bit wide scalar registers designated A0 to A15. Eachregister of global scalar register file 211 can be read from or writtento as 64-bits of scalar data. All scalar datapath side A 115 functionalunits (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225and D2 unit 226) can read or write to global scalar register file 211.Global scalar register file 211 may be read as 32-bits or as 64-bits andmay only be written to as 64-bits. The instruction executing determinesthe read data size. Vector datapath side B 116 functional units (L2 unit241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245 and P unit 246)can read from global scalar register file 211 via crosspath 117 underrestrictions that will be detailed below.

FIG. 4 illustrates D1/D2 local register file 214. There are 16independent 64-bit wide scalar registers designated D0 to D16. Eachregister of D1/D2 local register file 214 can be read from or written toas 64-bits of scalar data. All scalar datapath side A 115 functionalunits (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225and D2 unit 226) can write to global scalar register file 211. Only D1unit 225 and D2 unit 226 can read from D1/D2 local scalar register file214. It is expected that data stored in D1/D2 local scalar register file214 will include base addresses and offset addresses used in addresscalculation.

FIG. 5 illustrates L1/S1 local register file 212. The embodimentillustrated in FIG. 5 has 8 independent 64-bit wide scalar registersdesignated AL0 to AL7. The preferred instruction coding (see FIG. 13)permits L1/S1 local register file 212 to include up to 16 registers. Theembodiment of FIG. 5 implements only 8 registers to reduce circuit sizeand complexity. Each register of L1/S1 local register file 212 can beread from or written to as 64-bits of scalar data. All scalar datapathside A 115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1unit 224, D1 unit 225 and D2 unit 226) can write to L1/S1 local scalarregister file 212. Only L1 unit 221 and S1 unit 222 can read from L1/S1local scalar register file 212.

FIG. 6 illustrates M1/N1 local register file 213. The embodimentillustrated in FIG. 6 has 8 independent 64-bit wide scalar registersdesignated AM0 to AM 7. The preferred instruction coding (see FIG. 13)permits M1/N1 local register file 213 to include up to 16 registers. Theembodiment of FIG. 6 implements only 8 registers to reduce circuit sizeand complexity. Each register of M1/N1 local register file 213 can beread from or written to as 64-bits of scalar data. All scalar datapathside A 115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1unit 224, D1 unit 225 and D2 unit 226) can write to M1/N1 local scalarregister file 213. Only M1 unit 223 and N1 unit 224 can read from M1/N1local scalar register file 213.

FIG. 7 illustrates global vector register file 231. There are 16independent 512-bit wide vector registers. Each register of globalvector register file 231 can be read from or written to as 64-bits ofscalar data designated B0 to B15. Each register of global vectorregister file 231 can be read from or written to as 512-bits of vectordata designated VB0 to VB15. The instruction type determines the datasize. All vector datapath side B 116 functional units (L2 unit 241, S2unit 242, M2 unit 243, N2 unit 244, C unit 245 and P unit 246) can reador write to global vector register file 231. Scalar datapath side A 115functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1unit 225 and D2 unit 226) can read from global vector register file 231via crosspath 117 under restrictions that will be detailed below.

FIG. 8 illustrates P local register file 234. There are 8 independent64-bit wide registers designated P0 to P 15. Each register of P localregister file 234 can be read from or written to as 64-bits of scalardata. Vector datapath side B 116 functional units L2 unit 241, S2 unit242, C unit 245 and P unit 246 can write to P local register file 234.Only L2 unit 241, S2 unit 242 and P unit 246 can read from P localregister file 234. A commonly expected use of P local register file 234includes: writing one bit SIMD vector comparison results from L2 unit241, S2 unit 242 or C unit 245; manipulation of the SIMD vectorcomparison results by P unit 246; and use of the manipulated results incontrol of a further SIMD vector operation.

FIG. 9 illustrates L2/S2 local register file 232. The embodimentillustrated in FIG. 9 has 8 independent 512-bit wide vector registers.The preferred instruction coding (see FIG. 13) permits L2/S2 localregister file 232 to include up to 16 registers. The embodiment of FIG.9 implements only 8 registers to reduce circuit size and complexity.Each register of L2/S2 local vector register file 232 can be read fromor written to as 64-bits of scalar data designated BL0 to BL7. Eachregister of L2/S2 local register file 232 can be read from or written toas 512-bits of vector data designated VBL0 to VBL7. The instruction typedetermines the data size. All vector datapath side B 116 functionalunits (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245and P unit 24) can write to L2/S2 local vector register file 232. OnlyL2 unit 241 and S2 unit 242 can read from L2/S2 local register file 232.

FIG. 10 illustrates M2/N2/C local register file 233. The embodimentillustrated in FIG. 10 has 8 independent 512-bit wide vector registers.The preferred instruction coding (see FIG. 13) permits M2/N2/C localregister file 233 to include up to 16 registers. The embodiment of FIG.10 implements only 8 registers to reduce circuit size and complexity.Each register of M2/N2/C local register file 233 can be read from orwritten to as 64-bits of scalar data designated BM0 to BM7. Eachregister of M2/N2/C local register file 233 can be read from or writtento as 512-bits of vector data designated VBM0 to VBM7. All vectordatapath side B 116 functional units (L2 unit 241, S2 unit 242, M2 unit243, N2 unit 244, C unit 245 and P unit 246) can write to M2/N2/C localregister file 233. Only M2 unit 243, N2 unit 244 and C unit 245 can readfrom M2/N2/C local register file 233.

Crosspath 117 permits limited exchange of data between scalar datapathside A 115 and vector datapath side B 116. During each operational cycleone 64-bit data word can be recalled from global scalar register file A211 for use as an operand by one or more functional units of vectordatapath side B 116 and one 64-bit data word can be recalled from globalvector register file 231 for use as an operand by one or more functionalunits of scalar datapath side A 115. Any scalar datapath side A 115functional unit (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1unit 225 and D2 unit 226) may read a 64-bit operand from global vectorregister file 231. This 64-bit operand is the least significant bits ofthe 512-bit data in the accessed register of global vector register file231. Plural scalar datapath side A 115 functional units may employ thesame 64-bit crosspath data as an operand during the same operationalcycle. However, only one 64-bit operand is transferred from vectordatapath side B 116 to scalar datapath side A 115 in any singleoperational cycle. Any vector datapath side B 116 functional unit (L2unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245 and P unit246) may read a 64-bit operand from global scalar register file 211. Ifthe corresponding instruction is a scalar instruction, the crosspathoperand data is treated as any other 64-bit operand. If thecorresponding instruction is a vector instruction, the upper 448 bits ofthe operand are zero filled. Plural vector datapath side B 116functional units may employ the same 64-bit crosspath data as an operandduring the same operational cycle. Only one 64-bit operand istransferred from scalar datapath side A 115 to vector datapath side B116 in any single operational cycle.

Streaming engine 125 transfers data in certain restricted circumstances.Streaming engine 125 controls two data streams. A stream consists of asequence of elements of a particular type. Programs that operate onstreams read the data sequentially, operating on each element in turn.Every stream has the following basic properties. The stream data have awell-defined beginning and ending in time. The stream data have fixedelement size and type throughout the stream. The stream data have fixedsequence of elements. Thus programs cannot seek randomly within thestream. The stream data is read-only while active. Programs cannot writeto a stream while simultaneously reading from it. Once a stream isopened streaming engine 125: calculates the address; fetches the defineddata type from level two unified cache (which may require cache servicefrom a higher level memory); performs data type manipulation such aszero extension, sign extension, data element sorting/swapping such asmatrix transposition; and delivers the data directly to operand inputsof functional units of vector datapath B side 116. Streaming engine 125is thus useful for real-time digital filtering operations onwell-behaved data. Streaming engine 125 frees these memory fetch tasksfrom the corresponding CPU enabling other processing functions.

Streaming engine 125 provides the following benefits. Streaming engine125 permits multi-dimensional memory accesses. Streaming engine 125increases the available bandwidth to the functional units. Streamingengine 125 minimizes the number of cache miss stalls since the streambuffer bypasses level one data cache 123. Streaming engine 125 reducesthe number of scalar operations required to maintain a loop. Streamingengine 125 manages address pointers. Streaming engine 125 handlesaddress generation automatically freeing up the address generationinstruction slots and D1 unit 225 and D2 unit 226 for othercomputations.

CPU 110 operates on an instruction pipeline. Instructions are fetched ininstruction packets of fixed length further described below. Allinstructions require the same number of pipeline phases for fetch anddecode, but require a varying number of execute phases.

FIG. 11 illustrates the following pipeline phases: program fetch phase1110, dispatch and decode phases 1120 and execution phases 1130. Programfetch phase 1110 includes three stages for all instructions. Dispatchand decode phases 1120 include three stages for all instructions.Execution phase 1130 includes one to four stages dependent on theinstruction.

Program fetch phase 1110 includes program address generation stage 1111(PG), program access stage 1112 (PA) and program receive stage 1113(PR). During program address generation stage 1111 (PG), the programaddress is generated in the CPU and the read request is sent to thememory controller for the level one instruction cache L1I. During theprogram access stage 1112 (PA) the level one instruction cache L1Iprocesses the request, accesses the data in its memory and sends a fetchpacket to the CPU boundary. During the program receive stage 1113 (PR)the CPU registers the fetch packet.

In the illustrated example, instructions are always fetched sixteen32-bit wide slots, constituting a fetch packet, at a time. FIG. 12illustrates 16 instructions 1201 to 1216 of a single fetch packet. Fetchpackets are aligned on 512-bit (16-word) boundaries.

The execution of the individual instructions is partially controlled bya p bit in each instruction. This p bit is preferably bit 0 of the32-bit wide slot. The p bit determines whether an instruction executesin parallel with a next instruction. The p bits are scanned from lowerto higher address. If the p bit of an instruction is 1, then the nextfollowing instruction is executed in parallel with (in the same cycleas) that instruction. If the p bit of an instruction is 0, then the nextfollowing instruction is executed in the cycle after the instruction.

CPU 110 and level one instruction cache L1I 121 pipelines are de-coupledfrom each other. Fetch packet returns from level one instruction cacheL1I can take different number of clock cycles, depending on externalcircumstances such as whether there is a hit in level one instructioncache 121 or a hit in level two combined cache 130. Therefore programaccess stage 1112 (PA) can take several clock cycles instead of 1 clockcycle as in the other stages.

The instructions executing in parallel constitute an execute packet. Inthe preferred embodiment an execute packet can contain up to fourteeninstructions. No two instructions in an execute packet may use the samefunctional unit. A slot is one of five types: 1) a self-containedinstruction executed on one of the functional units of CPU 110 (L1 unit221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, D2 unit 226, L2unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245 and P unit246); 2) a unitless instruction such as a NOP (no operation) instructionor multiple NOP instruction; 3) a branch instruction; 4) a constantfield extension; and 5) a conditional code extension. Some of these slottypes will be further explained below.

Dispatch and decode phases 1110 include instruction dispatch toappropriate execution unit stage 1121 (DS), instruction pre-decode stage1122 (DC1), and instruction decode, operand reads stage 1123 (DC2).During instruction dispatch to appropriate execution unit stage 1121(DS), the fetch packets are split into execute packets and assigned tothe appropriate functional units. During the instruction pre-decodestage 1122 (DC1), the source registers, destination registers, andassociated paths are decoded for the execution of the instructions inthe functional units. During the instruction decode, operand reads stage1123 (DC2), more detailed unit decodes are done, as well as reading ofoperands from the register files.

Execution phases 1130 includes execution stages 1131 to 1135 (E1 to E5).Different types of instructions require different numbers of thesestages to complete their execution. These stages of the pipeline play animportant role in understanding the device state at CPU cycleboundaries.

During execute 1 stage 1131 (E1) the conditions for the instructions areevaluated and operands are operated on. As illustrated in FIG. 11,execute 1 stage 1131 may receive operands from a stream buffer 1141 andone of the register files shown schematically as 1142. For load andstore instructions, address generation is performed and addressmodifications are written to a register file. For branch instructions,branch fetch packet in PG phase 1111 is affected. As illustrated in FIG.11, load and store instructions access memory here shown schematicallyas memory 1151. For single-cycle instructions, results are written to adestination register file. This assumes that any conditions for theinstructions are evaluated as true. If a condition is evaluated asfalse, the instruction does not write any results or have any pipelineoperation after execute 1 stage 1131.

During execute 2 stage 1132 (E2) load instructions send the address tomemory. Store instructions send the address and data to memory.Single-cycle instructions that saturate results set the SAT bit in thecontrol status register (CSR) if saturation occurs. For 2-cycleinstructions, results are written to a destination register file.

During execute 3 stage 1133 (E3) data memory accesses are performed. Anymultiply instructions that saturate results set the SAT bit in thecontrol status register (CSR) if saturation occurs. For 3-cycleinstructions, results are written to a destination register file.

During execute 4 stage 1134 (E4) load instructions bring data to the CPUboundary. For 4-cycle instructions, results are written to a destinationregister file.

During execute 5 stage 1135 (E5) load instructions write data into aregister. This is illustrated schematically in FIG. 11 with input frommemory 1151 to execute 5 stage 1135.

FIG. 13 illustrates an example of the instruction coding 1300 offunctional unit instructions used by this invention. Each instructionconsists of 32 bits and controls the operation of one of theindividually controllable functional units (L1 unit 221, S1 unit 222, M1unit 223, N1 unit 224, D1 unit 225, D2 unit 226, L2 unit 241, S2 unit242, M2 unit 243, N2 unit 244, C unit 245 and P unit 246). The bitfields are defined as follows.

The creg field 1301 (bits 29 to 31) and the z bit 1302 (bit 28) areoptional fields used in conditional instructions. These bits are usedfor conditional instructions to identify the predicate register and thecondition. The z bit 1302 (bit 28) indicates whether the predication isbased upon zero or not zero in the predicate register. If z=1, the testis for equality with zero. If z=0, the test is for nonzero. The case ofcreg=0 and z=0 is treated as always true to allow unconditionalinstruction execution. The creg field 1301 and the z field 1302 areencoded in the instruction as shown in Table 1.

TABLE 1 Conditional creg z Register 31 30 29 28 Unconditional 0 0 0 0Reserved 0 0 0 1 A0 0 0 1 z A1 0 1 0 z A2 0 1 1 z A3 1 0 0 z A4 1 0 1 zA5 1 1 0 z Reserved 1 1 x x

Execution of a conditional instruction is conditional upon the valuestored in the specified data register. This data register is in theglobal scalar register file 211 for all functional units. Note that “z”in the z bit column refers to the zero/not zero comparison selectionnoted above and “x” is a do-not-care state. This coding can only specifya subset of the 16 global registers as predicate registers. Thisselection was made to preserve bits in the instruction coding. Note thatunconditional instructions do not have these optional bits. Forunconditional instructions these bits in fields 1301 and 1302 (28 to 31)are preferably used as additional opcode bits.

The dst field 1303 (bits 23 to 27) specifies a register in acorresponding register file as the destination of the instructionresults.

The src2 field 1304 (bits 18 to 22) specifies a register in acorresponding register file as the second source operand.

The src1/cst field 1305 (bits 13 to 17) has several meanings dependingon the instruction opcode field (bits 2 to 12 for all instructions andadditionally bits 28 to 31 for unconditional instructions). The firstmeaning specifies a register of a corresponding register file as thefirst operand. The second meaning is an immediate constant. Depending onthe instruction type, this is treated as an unsigned integer and zeroextended to a specified data length or is treated as a signed integerand sign extended to the specified data length.

The opcode field 1306 (bits 2 to 12) for all instructions (andadditionally bits 28 to 31 for unconditional instructions) specifies thetype of instruction and designates appropriate instruction options. Thisincludes unambiguous designation of the functional unit used andoperation performed. A detailed explanation of the opcode is beyond thescope of this invention except for the instruction options detailedbelow.

The s bit 1307 (bit 1) designates scalar datapath side A 115 or vectordatapath side B 116. If s=0, then scalar datapath side A 115 isselected. This limits the functional unit to L1 unit 221, S1 unit 222,M1 unit 223, N1 unit 224, D1 unit 225 and D2 unit 226 and thecorresponding register files illustrated in FIG. 2. Similarly, s=1selects vector datapath side B 116 limiting the functional unit to L2unit 241, S2 unit 242, M2 unit 243, N2 unit 244, P unit 246 and thecorresponding register file illustrated in FIG. 2.

The p bit 1308 (bit 0) marks the execute packets. The p-bit determineswhether the instruction executes in parallel with the followinginstruction. The p-bits are scanned from lower to higher address. If p=1for the current instruction, then the next instruction executes inparallel with the current instruction. If p=0 for the currentinstruction, then the next instruction executes in the cycle after thecurrent instruction. All instructions executing in parallel constitutean execute packet. In the illustrated examples, an execute packet cancontain up to fourteen instructions. Each instruction in an executepacket must use a different functional unit.

There are two different condition code extension slots. Each executepacket can contain one each of these unique 32-bit condition codeextension slots which contains the 4-bit creg/z fields for theinstructions in the same execute packet. FIG. 14 illustrates the codingfor condition code extension slot 0 and FIG. 15 illustrates the codingfor condition code extension slot 1.

FIG. 14 illustrates the coding for condition code extension slot 0having 32 bits. Field 1401 (bits 28 to 31) specify 4 creg/z bitsassigned to the L1 unit 221 instruction in the same execute packet.Field 1402 (bits 27 to 24) specify 4 creg/z bits assigned to the L2 unit241 instruction in the same execute packet. Field 1403 (bits 19 to 23)specify 4 creg/z bits assigned to the S1 unit 222 instruction in thesame execute packet. Field 1404 (bits 16 to 19) specify 4 creg/z bitsassigned to the S2 unit 242 instruction in the same execute packet.Field 1405 (bits 12 to 15) specify 4 creg/z bits assigned to the D1 unit225 instruction in the same execute packet. Field 1406 (bits 8 to 11)specify 4 creg/z bits assigned to the D2 unit 226 instruction in thesame execute packet. Field 1407 (bits 6 and 7) is unused/reserved. Field1408 (bits 0 to 5) are coded a set of unique bits (CCEX0, 011101) toidentify the condition code extension slot 0. Once this unique ID ofcondition code extension slot 0 is detected, the corresponding creg/zbits are employed to control conditional execution of any L1 unit 221,L2 unit 241, S1 unit 222, S2 unit 242, D1 unit 225 and D2 unit 226instruction in the same execution packet. These creg/z bits areinterpreted as shown in Table 1. Note that no execution packet can havemore than one instruction directed to a particular execution unit. Noexecute packet of instructions can contain more than one condition codeextension slot 0. Thus the mapping of creg/z bits to functional unitinstruction is unambiguous. Setting the creg/z bits equal to “0000”makes the instruction unconditional. Thus a properly coded conditioncode extension slot 0 can make some instructions conditional and someunconditional.

FIG. 15 illustrates the coding for condition code extension slot 1having 32 bits. Field 1501 (bits 28 to 31) specify 4 creg/z bitsassigned to the M1 unit 223 instruction in the same execute packet.Field 1502 (bits 27 to 24) specify 4 creg/z bits assigned to the M2 unit243 instruction in the same execute packet. Field 1503 (bits 19 to 23)specify 4 creg/z bits assigned to the C unit 245 instruction in the sameexecute packet. Field 1504 (bits 16 to 19) specify 4 creg/z bitsassigned to the N1 unit 224 instruction in the same execute packet.Field 1505 (bits 12 to 15) specify 4 creg/z bits assigned to the N2 unit244 instruction in the same execute packet. Field 1506 (bits 6 to 11) isunused/reserved. Field 1507 (bits 0 to 5) are coded a set of unique bits(CCEX1) to identify the condition code extension slot 1. Once thisunique ID of condition code extension slot 1 is detected, thecorresponding creg/z bits are employed to control conditional executionof any M1 unit 223, M2 unit 243, C unit 245, N1 unit 224 and N2 unit 244instruction in the same execution packet. These creg/z bits areinterpreted as shown in Table 1. If the corresponding instruction isconditional (includes creg/z bits) the corresponding bits in thecondition code extension slot 1 override the condition code bits in theinstruction. Note that no execution packet can have more than oneinstruction directed to a particular execution unit. No execute packetof instructions can contain more than one condition code extension slot1. Thus the mapping of creg/z bits to functional unit instruction isunambiguous. Setting the creg/z bits equal to “0000” makes theinstruction unconditional. Thus a properly coded condition codeextension slot 1 can make some instructions conditional and someunconditional.

There are two different constant extension slots. Each execute packetcan contain one each of these unique 32-bit constant extension slotswhich contains 27 bits to be concatenated as high order bits with the5-bit constant field 1305 to form a 32-bit constant. As noted in theinstruction coding description above only some instructions define thesrc1/cst field 1305 as a constant rather than a source registeridentifier. At least some of those instructions may employ a constantextension slot to extend this constant to 32 bits.

FIG. 16 illustrates the fields of constant extension slot 0. Eachexecute packet may include one instance of constant extension slot 0 andone instance of constant extension slot 1. FIG. 16 illustrates thatconstant extension slot 0 1600 includes two fields. Field 1601 (bits 5to 31) constitute the most significant 27 bits of an extended 32-bitconstant including the target instruction field 1305 as the five leastsignificant bits. Field 1602 (bits 0 to 4) are coded a set of uniquebits (CSTX0) to identify the constant extension slot 0. Constantextension slot 0 1600 can only be used to extend the constant of one ofan L1 unit 221 instruction, data in a D1 unit 225 instruction, an S2unit 242 instruction, an offset in a D2 unit 226 instruction, an M2 unit243 instruction, an N2 unit 244 instruction, a branch instruction, or aC unit 245 instruction in the same execute packet. Constant extensionslot 1 is similar to constant extension slot 0 except that bits 0 to 4are coded a set of unique bits (CSTX1) to identify the constantextension slot 1. Constant extension slot 1 can only be used to extendthe constant of one of an L2 unit 241 instruction, data in a D2 unit 226instruction, an S1 unit 222 instruction, an offset in a D1 unit 225instruction, an M1 unit 223 instruction or an N1 unit 224 instruction inthe same execute packet.

Constant extension slot 0 and constant extension slot 1 are used asfollows. The target instruction must be of the type permitting constantspecification. Instruction decoder 113 determines this from theinstruction opcode bits. The target instruction also includes oneconstant extension bit dedicated to signaling whether the specifiedconstant is not extended (preferably constant extension bit=0) or theconstant is extended (preferably constant extension bit=1). Ifinstruction decoder 113 detects constant extension slot 0 or constantextension slot 1, it further checks the other instructions within thatexecute packet for an instruction corresponding to the detected constantextension slot. A constant extension is made only if one correspondinginstruction has a constant extension bit equal to 1. An execute packetwith a constant extension slot and two corresponding instructions markedconstant extended (constant extension bit=1) is invalid.

FIG. 17 is a partial block diagram 1700 illustrating constant extension.FIG. 17 assumes that instruction decoder 113 detects a constantextension slot and a corresponding instruction in the same executepacket. Instruction decoder 113 supplies the 27 extension bits from theconstant extension slot and the 5 constant bits from the correspondinginstruction to concatenator 1701. Concatenator 1701 forms a single32-bit word from these two parts. This combined 32-bit word is suppliedto one input of multiplexer 1702. The 5 constant bits from thecorresponding instruction field 1305 supply a second input tomultiplexer 1702. Selection of multiplexer 1702 is controlled by thestatus of the constant extension bit. If the constant extension bit is 1(extended), multiplexer 1702 selects the concatenated 32-but input. Ifthe constant extension bit is 0 (not extended), multiplexer 1702 selectsthe 5 constant bits from the corresponding instruction field 1305.Multiplexer 1702 supplies this output to an input of sign extension unit1703.

Sign extension unit 1703 forms the final operand value from the inputfrom multiplexer 1703. Sign extension unit 1703 receives control inputsScalar/Vector and Data Size. The Scalar/Vector input indicates whetherthe corresponding instruction is a scalar instruction or a vectorinstruction. The functional units of data path side A 115 (L1 unit 221,S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225 and D2 unit 226) canonly perform scalar instructions. Any instruction directed to one ofthese functional units is a scalar instruction. Data path side Bfunctional units L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244 andC unit 245 may perform scalar instructions or vector instructions.Instruction decoder 113 determines whether the instruction is a scalarinstruction or a vector instruction from the opcode bits. P unit 246 mayonly perform scalar instructions. The Data Size may be 8 bits (byte B),16 bits (half-word H), 32 bits (word W) or 64 bits (double word D).

Table 2 lists the operation of sign extension unit 1703 for the variousoptions.

TABLE 2 Instruction Operand Constant Type Size Length Action ScalarB/H/W/D  5 bits Sign extend to 64 bits Scalar B/H/W/D 32 bits Signextend to 64 bits Vector B/H/W/D  5 bits Sign extend to operand size andreplicate across whole vector Vector B/H/W 32 bits Replicate 32-bitconstant across each 32-bit (W) lane Vector D 32 bits Sign extend to 64bits and replicate across each 64-bit (D) lane

Special vector predicate instructions use registers in predicateregister file 234 to control vector operations. In the currentembodiment all these SIMD vector predicate instructions operate onselected data sizes. The data sizes may include byte (8 bit) data, halfword (16 bit) data, word (32 bit) data, double word (64 bit) data, quadword (128 bit) data and half vector (256 bit) data. Each bit of thepredicate register controls whether a SIMD operation is performed uponthe corresponding byte of data. The operations of P unit 246 permit avariety of compound vector SIMD operations based upon more than onevector comparison. For example a range determination can be made usingtwo comparisons. A candidate vector is compared with a first vectorreference having the minimum of the range packed within a first dataregister. A second comparison of the candidate vector is made with asecond reference vector having the maximum of the range packed within asecond data register. Logical combinations of the two resultingpredicate registers would permit a vector conditional operation todetermine whether each data part of the candidate vector is within rangeor out of range.

L1 unit 221, S1 unit 222, L2 unit 241, S2 unit 242 and C unit 245 oftenoperate in a single instruction multiple data (SIMD) mode. In this SIMDmode the same instruction is applied to packed data from the twooperands. Each operand holds plural data elements disposed inpredetermined slots. SIMD operation is enabled by carry control at thedata boundaries. Such carry control enables operations on varying datawidths.

FIG. 18 illustrates the carry control. AND gate 1801 receives the carryoutput of bit N within the operand wide arithmetic logic unit (64 bitsfor scalar datapath side A 115 functional units and 512 bits for vectordatapath side B 116 functional units). AND gate 1801 also receives acarry control signal which will be further explained below. The outputof AND gate 1801 is supplied to the carry input of bit N+1 of theoperand wide arithmetic logic unit. AND gates such as AND gate 1801 aredisposed between every pair of bits at a possible data boundary. Forexample, for 8-bit data such an AND gate will be between bits 7 and 8,bits 15 and 16, bits 23 and 24, etc. Each such AND gate receives acorresponding carry control signal. If the data size is of the minimum,then each carry control signal is 0, effectively blocking carrytransmission between the adjacent bits. The corresponding carry controlsignal is 1 if the selected data size requires both arithmetic logicunit sections. Table 3 below shows example carry control signals for thecase of a 512 bit wide operand such as used by vector datapath side B116 functional units which may be divided into sections of 8 bits, 16bits, 32 bits, 64 bits, 128 bits or 256 bits. In Table 3 the upper 32bits control the upper bits (bits 128 to 511) carries and the lower 32bits control the lower bits (bits 0 to 127) carries. No control of thecarry output of the most significant bit is needed, thus only 63 carrycontrol signals are required.

TABLE 3 Data Size Carry Control Signals 8 bits (B) −000 0000 0000 00000000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 16 bits (H)−101 0101 0101 0101 0101 0101 0101 0101 0101 0101 0101 0101 0101 01010101 0101 32 bits (W) −111 0111 0111 0111 0111 0111 0111 0111 0111 01110111 0111 0111 0111 0111 0111 64 bits (D) −111 1111 0111 1111 0111 11110111 1111 0111 1111 0111 1111 0111 1111 0111 1111 128 bits −111 11111111 1111 0111 1111 1111 1111 0111 1111 1111 1111 0111 1111 1111 1111256 bits −111 1111 1111 1111 1111 1111 1111 1111 0111 1111 1111 11111111 1111 1111 1111

It is typical in the art to operate on data sizes that are integralpowers of 2 (2^(N)). However, this carry control technique is notlimited to integral powers of 2. One skilled in the art would understandhow to apply this technique to other data sizes and other operandwidths.

The data overlap in sliding window block sum calculations should beexploited when employing a vector SIMD engine for better performancecompared to a scalar engine. Summing elements within each column is easyvia a simple vector addition across rows. Summing elements selectivelywithin each row (horizontal sum) is difficult when employing a vectorSIMD engine. This represents a bottleneck in improving performance forsliding window block sum computation for vector SIMD engines.

FIG. 19 illustrates an example of a sliding window sum computation. FIG.19 illustrates portion 1900 of a video frame including plural pixels1901. In FIG. 19 each box represents the data of a corresponding pixel1901. FIG. 19 illustrates plural sliding windows each five pixels byseven pixels 1911, 1912, 1913, 1914 and 1915. Window 1911 centered aboutpixel P 1 has a vertical extent 1920 and a horizontal extent 1921.Window 1912 centered about pixel P 2 has a vertical extent 1920 and ahorizontal extent 1922. Window 1913 centered about pixel P 3 has avertical extent 1920 and a horizontal extent 1923. Window 1914 centeredabout pixel P 4 has a vertical extent 1920 and a horizontal extent 1914.Window 1915 centered about pixel P 5 has a vertical extent 1920 and ahorizontal extent 1925. Note the sliding window size illustrated in FIG.19 is exemplary only, larger or smaller windows are feasible.

The algorithm to be executed with the sliding windows 1911, 1912, 1913,1914 and 1915 is the sum of the pixel values in the whole slidingwindow. FIG. 20 illustrates one aspect of such a sliding window sum.FIG. 20 schematically illustrates a SIMD vector sum operation. Sourceregisters 2001 and 2002 each hold SIMD data in lanes 2010, 2020, 2030,2040, 2050, 2060, 2070 and 2080. Performing a SIMD sum operation resultsin the respective SIMD source values summed in destination register 2003as shown in FIG. 20. An initial step in summing all pixel values in asliding window is summing the rows. A first operation is a SIMD sum ofthe first two rows. A second operation is a SIMD sum of the prior sumand the third row. This continues performing a SIMD sum of the prior sumand the next row until reaching the last row in the sliding window. Theresulting SIMD sum is the sum of each column in the sliding window in acorresponding SIMD lane. FIG. 20 illustrates eight SIMD lanes inaccordance with the general practice of data sizes that are integralpowers of 2 (2^(N)) generally resulting in 2^(N) SIMD lanes. FIG. 20 isan example of a SIMD sum instruction that may be used for forming asliding window sum. The SIMD lane size should be selected at least asgreat as the pixel value size. SIMD lanes unused for the current slidingwindow sum may be zero filled or may be employed for another slidingwindow sum computation.

Prior art data processors do not include a horizontal SIMD suminstruction summing the values in each SIMD lane within a singleoperand. In absence of such a horizontal SIMD, addition of elements canoccur only across vectors and not within a vector. The prior artincludes two techniques to form the horizontal sum required to compute asliding window sum.

FIGS. 21 to 26 illustrate an example of a first of these prior arttechniques. This first prior art technique converts the horizontaladdition into a vertical addition. This conversion helps in slidingblock sum computation because it requires only simple vector addinstructions. FIG. 21 illustrates an exemplary 3 by 3 window withindividual elements A through I labeled for ease of reference. FIGS. 22to 24 illustrate three exemplary vector load operations used in thisprior art technique. FIG. 25 illustrates the values of the running sumupon initialization, at intermediate steps and the final value. FIG. 26is a flow chart 2600 outlining this prior art technique.

FIG. 21 illustrates an exemplary 3 by 3 window. This horizontal additionproblem requires calculation of A+B+C, D+E+F and G+H+I. FIG. 22illustrates the elements A through I disposed in memory in an order2200. This prior art technique begins with start block 2601 (FIG. 26).Next processing block 2601 initializes a running sum variable to zero.Variable 2501 (FIG. 25) illustrates an eight lane SIMD variable witheach SIMD lane set to zero. Depending on the relationship between theelement data size and the register and functional unit size of vectordata path B 116 (FIG. 1) there could be more or fewer SIMD lanes. Block2603 performs a next vector load. For the first iteration of this loop,the next vector load is the first vector load. FIG. 22 illustrates thefirst vector load of this example. The first vector load loads elementA, B, C, D, E, F, G and H into register 2201. This is treated as aneight lane SIMD variable the same as initialized running sum 2501. Block2604 sums the newly loaded vector quantity with the running sum.Intermediate running sum 2502 (FIG. 25) illustrates the result of sum.

Test block 2605 determines whether the just completed sum corresponds tothe last row. If not (No at test block 2605), then block 2606 leftshifts the vector load parameters by one element.

Block 2603 performs a next vector load at the newly calculated location.FIG. 23 illustrates the second vector load of this example which loadselement B, C, D, E, F, G, H and I into register 2301. Block 2604 sumsthe newly loaded vector quantity with the running sum. Intermediaterunning sum 2503 (FIG. 25) illustrates the result of sum. Note each SIMDlane includes the sum of corresponding lanes of registers 2201 and 2301.

Test block 2605 determines whether the just completed sum corresponds tothe last row. If not (No at test block 2605), then block 2606 leftshifts the vector load parameters by one element.

Block 2603 performs a next vector load at the newly calculated location.FIG. 24 illustrates the third vector load of this example which loadselement C, D, E, F, G, H, I and J (not within the original 3 by 3window) into register 2401. Block 2604 sums the newly loaded vectorquantity with the running sum. Running sum 2503 illustrates the resultof sum. Note each SIMD lane includes the sum of corresponding lanes ofregisters 2201, 2301 and 2401.

Test block 2605 determines whether the just completed sum corresponds tothe last row. If true (Yes at test block 2605), then process 2600 endsat end block 2607. The final running sum 2504 includes the horizontalsums needed. As illustrated in FIG. 25, SIMD lane 2511 includes thefirst row sum A+B+C, SIMD lane 2512 includes the second row sum D+E+Fand SIMD lane 2513 includes the third row sum G+H+I. Note that theelement distance between SIMD lane 2511 and SIMD land 2512 and betweenSIMD lane 2512 and SIMD lane 2513 is the block height (3 in thisexample).

Once horizontal sums corresponding to each element for the given blockwidth has been computed, next step involves addition of these horizontalsums for the given block height (vertical sum) to compute block sumcorresponding to each element. The resultant running sum of thehorizontal sums may are stored. The steps involved for sliding windowblock sum computation are similar to FIG. 26 except the shift in step2606 is by the block width. This aligns the correct elements of the SIMDresultant 2504 for column summing. This loop repeats for the blockheight resulting the sum of all window values.

This prior art technique has some serious shortcomings. The horizontalto vertical sum conversion trick, requires vector loads address offsetsof just one element. This results in almost no re-use of prior loadeddata. This does not effectively use the strength of a Vector SIMD enginein accelerating performance of the algorithm in question. This techniquerequires too many memory accesses due to this lack of re-use of loadeddata. Thus many memory operations are needed resulting in largeperformance overhead.

FIGS. 27 and 28 illustrate an example of a second of these prior arttechniques. FIG. 27 illustrates the relationship between element valuesin an image 2701 and an integral image 2702. The value of each elementin integral image 2702 is the sum the element and of all elements aboveand to the left of the element. Note window 2711 in image 2710. Thevalue of element 2711 in image 2710 is 1. The value of element 2722 inintegral image 2720 is 25. This is the sum of 5, 2, 3, 1, 5, 4, 2, 2and 1. All other values of integral image 2720 are derived similarly.

If there is an integral image available, this enables a quick andeffective way of calculating the sum of values (pixel values) in a givenimage or rectangular subset of a grid. Thus integral image can be usedfor calculating sliding window block sum as well. This calculation isillustrated in FIG. 28. This technique employs the integral values atthe four corners of the window I(A), I(B), I(C) and I(D). Using IntegralImage, block sum may be computed as depicted below:

Block Sum=I(C)+I(A)−I(B)−I(C)

Since a block sum can be computed using an integral image, the block sumcan also be used to compute a sliding window block sum. This approachhas many shortcomings. For large images, calculation of an integralimage is a time-consuming procedure. Thus, a sliding window block sumover the integral image results in large computation cost. Separatevector loads for integral image values at A, B, C and D are needed. Thisdoes not encourage data re-use. For example, points C and D for thecurrent block will be points A and B for another block. This algorithmdoes not make re-use of recalled data easy. The inherent nature ofintegral image prevents re-use of one block sum to calculate the nextadjacent block sum for the sliding window. Instead this algorithm callsfor a load of the integral image values for points A, B, C and D for theadjacent block. Integral image calculation results in higher memoryusage. The data type used for original image will typically not sufficefor integral images.

This invention employs an instruction called Vector Dot Product MaskPositive Negative (VDOTPMPN). This invention employs a sliding windowblock sum using instruction based selective horizontal addition. Anexample of a typical instruction that supports selective horizontaladdition is the VDOTPMPN instruction (Vector DOT Product Mask PositiveNegative). This instruction supports addition of elements within avector. The mask may be used to specify which elements within the vectorneed to be added to produce each element in the resultant sum vector.This instruction produces multiple such sums from a single vector. FIG.29 schematically illustrates the operation of the VDOTPMPN instruction.

The dot product of two vectors A=[A₁, A₂, . . . , A_(n)] and B=[B₁, B₂,. . . , B_(n)] is defined as:

${A \cdot B} = {{\sum\limits_{t = 1}^{n}{A_{t}B_{t}}} = {{A_{1}B_{1}} + {A_{2}B_{2}} + {\ldots \mspace{14mu} A_{n}B_{n}}}}$

where: n is the dimension of the vectors. The VDOTPMPN includes a maskoperand that is 1, 0 or −1 for each vector element. Proper use of thismask enables control over the number of terms and selection of additionor subtraction.

The horizontal sum corresponding to each element for the given blockwidth can be calculated easily using the VDOTPMPN instruction. The maskneeds to be prepared such that the right set of elements needs to beselected for this purpose. For a sliding window block sum, the blockwidth remains the same throughout the image. Thus the mask can bepre-computed and stored. The mask preparation therefore does not resultin any performance loss.

FIG. 30 is a flow chart illustrating process 3000 of this invention.Process 3000 begins at start block 3001. Process 3000 precalculates themasks as noted above in block 3002. Process 3000 then loads the nextvector of picture elements into a vector register in block 3003. For thefirst pass through the loop, the next vector of picture elements is thefirst vector of picture elements.

Process 3000 then calculates row sums using a VDOTPMPN instruction inblock 3004. This process is illustrated in FIG. 31 for an example ofeight element vector length for a block sum of a 3×3 block. Vector 3101illustrates eight elements packed in the vector register I_(aa), I_(ab). . . I_(ah). Each element of resultant vector 3110 is the sum of threeelements of vector 3101. As illustrated in FIG. 31: element H_(ab) isthe sum of I_(aa), I_(ab) and I_(ac); element H_(ac) is the sum ofI_(ab), I_(ac) and I_(ad); element H_(ad) is the sum of I_(ac), I_(ad)and I_(ae); element H_(ae) is the sum of I_(ad), I_(ae) and I_(af);element H_(af) is the sum of I_(ae), I_(af) and I_(ag); and elementH_(ag) is the sum of I_(af), I_(ag) and I_(ah). As required each row sumspans the three column image size.

Decision block 3005 determines if the row sums of the entire image havebeen processed. The image size is typically larger than the block sizeof the block sum. Thus more than one pass using the VDOTPMPN instruction(block 3004) is generally required. If the image has not been completelytraversed (No at decision block 3005), then process 3000 advances toblock 3003 to load the next vector of elements.

If the image has been completely traversed (Yes at decision block 3005),then process 3000 advances to block 3006 to compute the column sums.This process is illustrated in FIG. 32. Vectors 3201, 3202 and 3203 arerow sums from block 3004. Vector 3211 is a SIMD sum of the correspondingrow sums. Though only a single operation is shown in FIG. 32, in apractical embodiment a SIMD sum instruction will generally support onlytwo input operands. The larger sum illustrated in FIG. 32 would beformed from plural SIMD add operations such as:

Intermediate Sum=SIMD ADD (3201+3202); and

Final Sum=SIMD Add (Intermediate Sum+3203).

The number of intermediate sums needed depends on the column with of theblock to be summed. For sliding window sums data re-use is maximizedusing a single running sum. SIMD addition is performed for each row ofhorizontal sums is performed to the running sum, the horizontal sum forthe row just above the block height for the current element may besubtracted from the running sum. Upon completion of the column sumsprocess 3000 is complete and ends at end block 3007.

This solution is simple and straightforward and has the followingadvantages. The row sum calculation is simplified using Horizontal SIMD.Because the block size is fixed for the sliding window, the weights tobe applied for VDOTPMPN instruction can be pre-computed. This solutionencourages maximum re-use of once loaded data. This solution thusreduces the frequency of memory accesses.

Using VDOTPMPN instruction restricts calculation to less than the SIMDwidth of data at a time depending on the block size required. Thus thissolution produces less data than it consumes. This restricts operationto less than the full vector bandwidth. The strengths of this solutionmore than offsets the shortcoming, resulting in significantly enhancedperformance for computation of sliding window block sum.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of what may be claimed, but ratheras descriptions of features that may be specific to particularembodiments. Certain features that are described in this specificationin the context of separate embodiments can also be implemented incombination in a single embodiment. Conversely, various features thatare described in the context of a single embodiment can also beimplemented in multiple embodiments separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults unless such order is recited in one or more claims. In certaincircumstances, multitasking and parallel processing may be advantageous.Moreover, the separation of various system components in the embodimentsdescribed above should not be understood as requiring such separation inall embodiments.

What is claimed is:
 1. A method for computing a sliding window block sumfor a matrix of picture elements having a size M×N, the methodcomprising: using a processor to: for each picture element in thematrix: determine a block sum for a window having a size m×n around thepicture element, wherein M>m and N>n, and wherein determining the blocksum comprises: for each row of the window, performing a vector dotproduct operation based on a vector of packed picture elementscorresponding to the row and a mask that is determined based at leastpartially on a horizontal size of the window to determine a respectivevector of masked horizontal element sums; and after determining therespective vector of masked horizontal element sums for each row of thewindow, performing a vector single instruction multiple data (SIMD)addition operation to sum the respective vectors of masked horizontalelement sums for each row of the window to determine the block sum ofthe window that corresponds to the picture element; and replace eachpicture element in the matrix with the block sum of the windowcorresponding to the picture element to obtain the sliding window blocksum of the matrix.
 2. The method of claim 1, wherein: M is an integernumber corresponding to a width of the matrix; N is an integer numbercorresponding to a height of the matrix; m is an integer numbercorresponding to a width of the window; and n is an integer numbercorresponding to a height of the window.
 3. The method of claim 1,wherein: M represents the number of columns in the matrix; N representsthe number of rows in the matrix; m represents the number of columns inthe window; and n represents the number of rows in the window.
 4. Themethod of claim 1, wherein the mask is determined based additionally ona vector size of the packed picture elements.
 5. The method of claim 1,wherein the mask comprises a respective bit for each picture element. 6.The method of claim 5, wherein the respective bit for each pictureelement is selectable as either a first value that indicates an additionoperation, a second value that indicates a subtraction operation, and athird value that indicates the picture element should not be selected.7. The method of claim 1, wherein summing the vectors of maskedhorizontal element sums comprises: summing first and second vectors ofmasked horizontal element sums to form a first sum; and then summing anadditional vector of masked horizontal element sums with the first sumforming a second sum.
 8. The method of claim 7, wherein, when n >3, thefirst and second sums are intermediate sums.
 9. The method of claim 7,wherein, when the additional vector of masked horizontal element sumscorresponds to a last row of the window, the second sum is the block sumof the window.
 10. The method of claim 1, wherein the processorcomprises a digital signal processor.
 11. The method of claim 1, whereinthe window is centered around the picture element.
 12. An electronicdevice comprising: a processor; and a memory to store image data, theimage data including a matrix of picture elements having a size M×N, andinstructions for determining a sliding window block sum of the matrixthat, when executed by the processor, causes the processor to: for eachpicture element in the matrix: determine a block sum for a window havinga size m×n around the picture element, wherein M>m and N>n, and whereindetermining the block sum comprises: for each row of the window,performing a vector dot product operation based on a vector of packedpicture elements corresponding to the row and a mask that is determinedbased at least partially on a horizontal size of the window to determinea respective vector of masked horizontal element sums; and afterdetermining the respective vector of masked horizontal element sums foreach row of the window, performing a vector single instruction multipledata (SIMD) addition operation to sum the respective vectors of maskedhorizontal element sums for each row of the window to determine theblock sum of the window that corresponds to the picture element; andreplace each picture element in the matrix with the block sum of thewindow corresponding to the picture element to obtain the sliding windowblock sum of the matrix.
 13. The electronic device of claim 12, wherein:M is an integer number that represents the number of columns in thematrix; N is an integer number that represents the number of rows in thematrix; m is an integer number that represents the number of columns inthe window; and n is an integer number that represents the number ofrows in the window.
 14. The electronic device of claim 12, wherein themask is determined based additionally on a vector size of the packedpicture elements.
 15. The electronic device of claim 12, wherein themask comprises a respective bit for each picture element, wherein therespective bit for each picture element is selectable as either a firstvalue that indicates an addition operation, a second value thatindicates a subtraction operation, and a third value that indicates thepicture element should not be selected.
 16. The electronic device ofclaim 12, wherein the processor comprises a digital signal processor.17. The electronic device of claim 12, wherein the processor isconfigured to sum the vectors of masked horizontal element sums by:summing first and second vectors of masked horizontal element sums toform a first sum; and then summing an additional vector of maskedhorizontal element sums with the first sum forming a second sum.
 18. Theelectronic device of claim 17, wherein, when n>3, the first and secondsums are intermediate sums.
 19. The electronic device of claim 17,wherein, when the additional vector of masked horizontal element sumscorresponds to a last row of the window, the second sum is the block sumof the window.
 20. The electronic device of claim 12, wherein the windowhas a size of m=3 and n=3.